Skip to content

Add "Create forecast" button to sensor page#1985

Open
Copilot wants to merge 30 commits intomainfrom
copilot/add-forecasting-button
Open

Add "Create forecast" button to sensor page#1985
Copilot wants to merge 30 commits intomainfrom
copilot/add-forecasting-button

Conversation

Copy link
Contributor

Copilot AI commented Feb 24, 2026

image

Adds a "Create forecast" side-panel to the sensor page, letting users trigger a regression forecast with a single click — no CLI or API knowledge required.

Behaviour

  • Panel is visible only to users with permission to record data on sensors
  • Forecast duration defaults to 48 hours (driven by FLEXMEASURES_PLANNING_HORIZON config) and can be adjusted up to 7 days via a duration input field
  • Button is enabled only when the sensor has ≥ 2 days of historical data; otherwise shown disabled with an explanation
  • Info icon tooltip shows the humanized default duration (e.g. "2 days") from the server config
  • Clicking triggers POST /api/v3_0/sensors/{id}/forecasts/trigger (using the selected duration, no start → defaults to now), polls GET /api/v3_0/sensors/{id}/forecasts/{uuid} every 3 s, surfaces progress via Toast messages, and refreshes the chart data on success (without a full page reload)
  • A "More options ↗" link next to the button opens the relevant OpenAPI docs endpoint in a new tab

Changes

View (ui/views/sensors.py)

  • Checks create-children permission via user_can_create_children(sensor) before calling get_timerange (DB short-circuit for users without permission)
  • Reads FLEXMEASURES_PLANNING_HORIZON from app config and passes the ISO string, humanized string, and integer days to the template

Template (ui/templates/sensors/index.html)

  • New sidepanel-container block matching the style of the existing "Upload data" panel
  • Info icon tooltip uses the humanized planning horizon from config
  • Duration (days) number input pre-populated from the server config, capped at 7 days
  • "More options ↗" link to the OpenAPI trigger endpoint (opens in new tab), placed left of the "Trigger job" button
  • JS reads the duration input and sends the ISO duration to the API; on success dispatches sensorsToShowUpdated to reload chart data via the existing embedAndLoad handler in graphs.html
  • JS block conditionally emitted by Jinja2 (only when both permission and data flags are true)

Tests (ui/tests/test_sensor_views.py)

New test file with 8 cases: page access, 404, login redirect, panel visibility per permission level (own-account user / admin / other-account user), button state at the 2-day boundary, and a mock-based guard verifying get_timerange is not called without permission.

Documentation

  • documentation/views/sensors.rst — new "Creating a forecast" section with cross-reference; updated to use plain-language permissions and config-based duration
  • documentation/features/forecasting.rst — new "Forecasting via the UI" subsection alongside the existing CLI and API sections; plain-language permissions, config-based default duration, and a bridge sentence directing users to the API for more control
  • documentation/changelog.rst — entry added
Original prompt

This section details on the original issue you should resolve

<issue_title>Forecasting button</issue_title>
<issue_description>A button should appear on the sensor page, which will create a simple forecast. This can make it easy to demonstrate this feature in demonstrations and quickly build examples.

At first, this is limited to the most basic version (vision: as little effort as just the button click).

  • forecast 24 hours
  • only regressive
  • only active when at least two days of data is given and if the users has permissions to create-children on the sensor

UX:

  • The button calls the endpoint per Ajax, and polls the job every 3 seconds, to give feedback in form of a Toast message.
  • If data is available we could reload the graph.
  • An info icon explains the functionality and limitations.</issue_description>

<agent_instructions>FM v0.31.0 will see new forecasting API endpoints, with all timing parameters getting sensible defaults for ease of use [POST] /sensors/(id)/forecasts/trigger <api/v3_0.html#post--api-v3_0-sensors-id-forecasts-trigger>_ and [GET] /sensors/(id)/forecasts/(uuid) <api/v3_0.html#get--api-v3_0-sensors-id-forecasts-uuid>_ to forecast sensor data [see PR #1813 <https://www.github.com/FlexMeasures/flexmeasures/pull/1813>, PR #1823 <https://www.github.com/FlexMeasures/flexmeasures/pull/1823>, PR #1917 <https://www.github.com/FlexMeasures/flexmeasures/pull/1917>_ and PR #1982 <https://www.github.com/FlexMeasures/flexmeasures/pull/1982>_]

Please add a forecasting button on the UI's sensor page accordingly, maybe in proximity to and in a similar visual fashion as the "Upload data" side-panel, which also deals with recording new time series data on the sensor.

Employ your agent team and make sure to run the test specialist, the docs specialist and the coordinator. I suspect the coordinator might want to start a ui-specialist.md with the lessons learned. Make sure to tell it to consider that.

You must check precommit hooks and the full test suite passes before asking for my review.</agent_instructions>

Comments on the Issue (you are @copilot in this section)

Custom agent used: review-lead
Orchestrates team of specialist agents for comprehensive code reviews and synthesizes their findings into unified recommendations


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

@read-the-docs-community
Copy link

read-the-docs-community bot commented Feb 24, 2026

Documentation build overview

📚 flexmeasures | 🛠️ Build #31762936 | 📁 Comparing 5c09b1e against latest (c6bb60f)


🔍 Preview build

Show files changed (8 files in total): 📝 8 modified | ➕ 0 added | ➖ 0 deleted
File Status
changelog.html 📝 modified
genindex.html 📝 modified
_autosummary/flexmeasures.data.models.forecasting.pipelines.train_predict.html 📝 modified
_autosummary/flexmeasures.data.models.forecasting.utils.html 📝 modified
_autosummary/flexmeasures.data.models.generic_assets.html 📝 modified
api/v3_0.html 📝 modified
features/forecasting.html 📝 modified
views/sensors.html 📝 modified

Copilot AI added 2 commits February 24, 2026 18:37
Context:
- PR #1985 added the 'Create Forecast' button to the sensor page UI
- No agent existed to encode Flask/Jinja2/JS UI patterns for FlexMeasures
- Patterns were worked out from scratch during the session

Change:
- Created .github/agents/ui-specialist.md covering:
  - Side-panel pattern (sidepanel-container / left-sidepanel-label hierarchy)
  - Permission-gating idiom: always short-circuit DB calls behind permission check
  - JS fetch → poll loop → Toast → reload pattern with correct status-code handling
  - Toast/spinner setup and teardown discipline
  - Disabled-button vs. hidden-panel distinction
  - Marshmallow data_key awareness for JS POST payloads
  - UI test checklist (access, permissions, data availability, boundary, not-called guard)
  - Known architectural debt (absent CSRF, session-expiry in poll loop, type hint gap)
Context:
- PR #1985 established concrete UI patterns (side panel, JS poll loop,
  permission-gated view logic) that the agent system had no coverage for
- A new UI Specialist agent was created to own those patterns
- Coordinator roster and domain knowledge should reflect this

Change:
- Added UI Specialist to the agent roster (item 9)
- Added 'UI Development Patterns' subsection to Domain Knowledge with:
  - Jinja2 side-panel template snippet
  - Python view short-circuit data-guard pattern
  - JS fetch/poll/Toast/reload skeleton
  - Agent responsibility table for UI-touching PRs
Copilot AI changed the title [WIP] Add forecasting button to sensor page Add one-click "Create forecast" button to sensor page Feb 24, 2026
Copilot AI requested a review from Flix6x February 24, 2026 18:42
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…nit__() missing 1 required argument: 'connection'`

Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
@Flix6x Flix6x marked this pull request as ready for review February 25, 2026 10:15
…docs updates

Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com>
Copilot AI changed the title Add one-click "Create forecast" button to sensor page Add "Create forecast" button to sensor page Feb 25, 2026
Copilot AI requested a review from Flix6x February 25, 2026 10:48
…d updating

Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
@Flix6x Flix6x added this to the 0.31.0 milestone Feb 25, 2026
@Flix6x Flix6x requested a review from nhoening February 25, 2026 11:30
@nhoening
Copy link
Contributor

nhoening commented Mar 3, 2026

As we go though a few steps, Toast messages help the user understand that.

Can somebody who has run this tell me how long the spinner is there now? Maybe until the triggering of the job has returned a success or failure? For at least that a spinner makes sense. It also seems to make sense not to allow hitting the same button for the same sensor again until the actual job outcome is in. I would go with that, actually for now.

The cooldown idea is also relevant, but we need to discuss this better. Probably rate-limiting on the server is the best use of our time. (and for that we need some decisions and data modeling on account level)

@Flix6x
Copy link
Contributor

Flix6x commented Mar 4, 2026

Sorry @joshuaunity my bad. When I read your comment, I interpreted it as a suggestion to add a spinner. I did not realize the spinner was already there. (Even though you added a screenshot, I thought this was a quick design of your own.) I now realize the spinner is already there, and to answer @nhoening, it keeps being shown until the forecasting job is done, which takes about 5 to 20 seconds or so, depending on the number of days selected.

So your suggestion was rather to change the styling of the spinner. I also now see we use these bootstrap spinners elsewhere already, so I'll gladly take up your suggestion, thanks!

Signed-off-by: F.N. Claessen <claessen@seita.nl>
@Flix6x Flix6x requested a review from joshuaunity March 4, 2026 10:04
@Flix6x
Copy link
Contributor

Flix6x commented Mar 4, 2026

I believe this PR also closes #1782.

@Flix6x Flix6x linked an issue Mar 4, 2026 that may be closed by this pull request
Copy link
Contributor

@nhoening nhoening left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a review from me testing this out, I see two issues:

First, when there is not enough data, I cannot read the text on the button very well:

Image

Second, maybe the training period needs some tweaking?
In my example, I had uploaded two days of data, which is a week old or so. The forecasting went okay, but it forecast zeroes, maybe due to the traning period being long (Training cycle from 2026-02-09 11:15:00+01:00 to 2026-03-11 11:15:00+01:00 started ....)
The image below can illustrate it (the forecasts start 11 march). I don't think the UX around this is useful enough yet. Happy to discuss.

Image

Finally, the button is not easy to find right now. Maybe we can ask Copilot in a follo-up PR if the sliding sidebar elements can stop taking in vertical space when they are closed.

@Flix6x
Copy link
Contributor

Flix6x commented Mar 11, 2026

In my example, I had uploaded two days of data, which is a week old or so. The forecasting went okay, but it forecast zeroes [...]

The forecast doesn't look particularly wrong to me given the training data. Maybe test with a more complete history?

Finally, the button is not easy to find right now. Maybe we can ask Copilot in a follow-up PR if the sliding sidebar elements can stop taking in vertical space when they are closed.

This side panels have grown over time to the point where the sensor page is vertically scrollable. I propose that we get rid of the headers on the side panels, which I find quite redundant. I think this should reduce enough vertical spacing. [update: alas, even without the headers I can still scroll the sensor page on my laptop]

@nhoening
Copy link
Contributor

The forecast doesn't look particularly wrong to me given the training data. Maybe test with a more complete history?

I only added two days of training data, which are non-zero. I expect the model to be trained on that, not interpolated zeroes of a one-month period around my actual data.

Signed-off-by: F.N. Claessen <claessen@seita.nl>
@Flix6x
Copy link
Contributor

Flix6x commented Mar 11, 2026

So your expectation is that the UI automatically finds out the available data range (limited within the default training period of the last 30 days?), and selects that as the training period? I don't think our API even supports setting a training end that is different than the predict start.

Flix6x added 2 commits March 11, 2026 12:24
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
@Flix6x Flix6x requested a review from nhoening March 11, 2026 11:32
Flix6x added 3 commits March 11, 2026 14:19
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Copy link
Contributor

@nhoening nhoening left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making a bit better use of the space. I still believe we should not have all this vertical space between the buttons, and I started a chatbot session to find a better way. Should be a follow-up PR.

I would address the UX / expectation problem in two ways (I know we want to keep this feature as simple as we can for now, but if users have a bad time, we should not ship it at all, or only show it to admins):

  • The help text should more precisely explain what the forecaster is doing, for instance that the training window is fixed and not adjusted to available data.
  • I can imagine that users often have data sets (and often in the phase when this button is interesting, they upload them), but they are not perfectly current, i.e. often there is a gap of a day or more. I suggest that the code forecasts by definition from the end of available data. We can discuss limiting that data to beliefs with horizon <= 0, and/or to add a dropdown for choosing between "forecasts start now" and "forecasts start at the end of available data")

Signed-off-by: F.N. Claessen <claessen@seita.nl>
@Flix6x
Copy link
Contributor

Flix6x commented Mar 11, 2026

I suggest that the code forecasts by definition from the end of available data.

I'll assume you mean the UI code, which is what I would choose (and not, say the API code). I'd prefer to just expand the form with a few basic customisation options, such as the ability to set your own start time.

Signed-off-by: F.N. Claessen <claessen@seita.nl>
@nhoening
Copy link
Contributor

I suggest that the code forecasts by definition from the end of available data.

I'll assume you mean the UI code, which is what I would choose (and not, say the API code). I'd prefer to just expand the form with a few basic customisation options, such as the ability to set your own start time.

Okay, the forecasting API endpoint has a start field, which we should use. Understood.

Options I see:

  • The dialogue gets the ability to choose the start time. Actually not very easy (adding a date picker).
  • We implement my idea of the dropdown in the dialogue. For this, the frontend needs to know the event_start of the most recent belief (so it can set the start of forecasts to that time if the user chooses that option) - and it can atually get it from the stats on the same page ("Last recorded")! 👍 Potential problem: uploading new data doesn't update the stats (nor the graphs). That is something that should actually happen, maybe a bug?

Flix6x added 2 commits March 11, 2026 16:45
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
@Flix6x
Copy link
Contributor

Flix6x commented Mar 11, 2026

image

The datetime selector might show up differently for you, because it depends on the user's browser and locale, see https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/input/datetime-local.

@nhoening
Copy link
Contributor

Okay, can we preset the datetime selector to the time f the last event, since it is already available, at least?

I want to keep it possible for this feature that the user does not have to think a lot.

Flix6x added 2 commits March 11, 2026 17:24
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
@Flix6x Flix6x requested a review from nhoening March 11, 2026 16:28
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Forecasting button Fetch new sensor data when saving sensors_to_show

4 participants